Transformation Function Insertion for Dynamically Displayed Tracer Data

ABSTRACT

A visualization system for a tracer may include a processing pipeline that may generate tracing data, preprocess the data, and visualize the data. The preprocessing step may include a mechanism to process user-defined expressions or other executable code. The executable code may perform various functions including mathematical, statistical, aggregation with other data, and others. The preprocessor may perform malware analysis, test the functionality, then implement the executable code. A user may be presented with an editor or other text based user interface component to enter and edit the executable code. The executable code may be saved and later recalled as a selectable transformation for use with other data streams.

BACKGROUND

Message passing computational environments operate by having independentprocessing elements, such as threads or other computational components,pass messages from one element to another during execution. The messagespassed between components may contain data and other information thatmay be consumed by the recipient.

SUMMARY

A visualization system for a tracer may include a processing pipelinethat may generate tracing data, preprocess the data, and visualize thedata. The preprocessing step may include a mechanism to processuser-defined expressions or other executable code. The executable codemay perform various functions including mathematical, statistical,aggregation with other data, and others. The preprocessor may performmalware analysis, test the functionality, then implement the executablecode. A user may be presented with an editor or other text based userinterface component to enter and edit the executable code. Theexecutable code may be saved and later recalled as a selectabletransformation for use with other data streams.

A force directed graph may serve as a part of a user control for atracer. The tracer may collect data while monitoring an executingapplication, then the data may be processed and displayed on a forcedirected graph. A user may be able to select individual nodes, edges, orother elements, then cause the tracer to change what data may becollected. The user may be able to select individual nodes, edges, orgroups of elements on the graph, then perform updates to the tracerusing the selected elements. The selection mechanisms may includeclicking and dragging a window to select nodes that may be related, aswell as selecting from a legend or other grouping.

A force directed graph may display time series data using a set ofplayback controls to pause, play, reverse, fast forward, slow down, orotherwise control the display of the time series data. The playbackcontrols may be used in a real time or near real time application towhich data sets are displayed and the speed with which the data sets maybe displayed. In one architecture, the force directed graph may bedeployed using a rendering engine that receives data and renders thedata into a graph. A playback controller may send updates to therendering engine according to user inputs from the playback controls.

A message passing compute environment may be visualized by illustratingmessages passed within the environment. The messages may contain dataconsumed by a function or other computational element, and may be usedto launch or spawn various computational elements. One visualization maybe a force directed graph that has each function as a node, withmessages passed as edges of the graph. In some embodiments, the edgesmay display the number of messages, quantity of data, or other metric byshowing the edges as wider or thinner, or by changing the color of thedisplayed edge. The nodes may be illustrated with different colors,size, or shape to show different aspects. Some embodiments may have amechanism for storing and playing back changes to the graph over time.

A force directed graph may display recent activities of a messagepassing system as highlighted features over a larger graph. The forcedirected graph may display a superset of nodes and edges representingprocesses and message routes, then display recent activities ashighlighted elements within the larger superset. The highlightedelements may display messages passed or computation performed during arecent time element of a time series. In some embodiments, the effectsof activities may be displayed by decaying the highlighted visualelements over time.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings,

FIG. 1 is a diagram illustration of an embodiment showing a forcedirected graph.

FIG. 2 is a diagram illustration of an embodiment showing an environmentfor data collection and display using a graph.

FIG. 3 is a flowchart illustration of an embodiment showing a method forgathering data.

FIG. 4 is a flowchart illustration of an embodiment showing a method foraggregating data prior to visualization.

FIG. 5 is a flowchart illustration of an embodiment showing a method fordeploying and updating a graph.

FIG. 6 is a flowchart illustration of an embodiment showing a method forgenerating traces of objects on a graph.

FIG. 7 is a diagram illustration of an embodiment showing a sample forcedirected graph displaying a time series.

FIGS. 8A, 8B, and 8C are a sequence of diagram illustrations of anembodiment showing a selection mechanism with a force directed graph.

FIGS. 9A, 9B, and 9C are a sequence of diagram illustrations of anembodiment showing a second selection mechanism with a force directedgraph.

FIG. 10 is a flowchart illustration of an embodiment showing a methodcontrolling a tracer through an interactive graph.

FIG. 11 is a diagram illustration of an embodiment showing a networkenvironment for visualizing trace data.

FIG. 12 is a diagram illustration of an embodiment showing a method forvisualizing trace data with transformations.

FIG. 13 is a diagram illustration of an embodiment showing a sample userinterface with a transformation editor.

FIG. 14 is a diagram illustration of an embodiment showing a networkenvironment with transformations.

FIG. 15 is a flowchart illustration of an embodiment showing a methodfor controlling a display using a data browser.

DETAILED DESCRIPTION

Graphs for Visualizing a Message Passing Compute Environment

A message passing compute environment may be visualized by showinggraphs of the messages passed between compute elements. The graphs mayshow the compute elements as nodes, with messages as edges of the graph.One type of such a visualization may be a force directed graph.

The visualization may illustrate different features of the data, such asthe number of messages, quantity of data, direction of messages, orother features as line widths, colors, or other visual elements. In thecase of a force directed graph, the forces between elements mayrepresent such data features.

The nodes of a graph may represent compute elements. The computeelements may be any executable code, device, or other element that maysend or receive a message. The nodes may be illustrated with differentsizes, colors, shapes, or other features to illustrate the amount ofcomputational time consumed, frequency of calling, membership in agroup, interaction with other elements, or other data items.

The visualization may be performed using a sequence of data sets, whereeach data set may be collected over time. In such embodiments, a graphmay expand, contract, and change shape as an application executes. Suchembodiments may be capable of storing and playing back the sequence ofdata sets. In some cases, such playback may be slowed down or sped up toillustrate changes during execution.

The visualization system may have an instrumentation system that gathersmessage information during execution, then processes or formats theinformation for display. The display system may generate the graphs anddisplay the graphs for a user. In some cases, the graphs may beinteractive, where the user may be able to probe the graphs to gainadditional insight. In one example, a user may be able to click on anode to find details about the node, such as the node name, performancemetrics regarding the node, or other information.

The visualization system may be used to monitor and display messagespassed within a single device, as well as embodiments where messages arepassed between devices. For example, some functional languages may passmessages between processes that may execute on a single processor oracross several processors within a single device. In another example, ahigh performance computing system may combine processors located on manydifferent devices to execute a large application. Such an applicationmay be visualized by showing all of the messages passed from device todevice, as well as from one process to another within each individualdevices, for example.

Force Directed Graph for Time Series Data with Highlighting

A force directed graph may display time series data by maintaining asuperset of nodes and edges, and displaying recent activity byhighlighting those elements within the graph representing the recentactivity. The superset of nodes and edges may be created by capturingeach node and edge that may be defined through the time series andmaintaining the superset during playback or display of a time series.

Recent activity may be overlaid on the superset of elements byhighlighting those elements that represent the activity, while showingat least some of the superset of nodes and edges in a non-highlightedfashion. In one style of such a visualization, the superset of nodes andedges may be presented in a greyed-out fashion while recently activenodes and edges may be presented in a colored manner.

The recent activity may be illustrated as fading or dissolving bycausing an element to decrease in highlighting for successive timeperiods after being active. Such a visual decay may highlight an activeelement yet keep a visual cue for a certain number of time slices, andmay be useful in cases where the time slices are short enough thatactivity in a single time slice may not be fully comprehended.

Visualization of Time Series Data with Force Directed Graph

A dynamic visualization of time series data may be rendered in a forcedirected graph. The time series data may include data sets thatrepresent a state of a system at any given time. The visualization mayillustrate the state changes as time progresses.

The visualization may have a set of controls that allow a user to moveforward and backwards through the data sets. The controls may allow theuser to control playback of the data. In some cases, the data may bepresented in a normal-time basis where the playback may correspond withthe speed of the data collection. In other cases, the playback may besped up or slowed down with respect to the periodicity in which the datawere collected.

An architecture for a visualization system may have a visualizer thatmay be bound to a data source. The visualizer may display the forcedirected graph, including rendering any animated motion of the forces.The controls may configure a data browser that may select the data setsto present, which may be transferred to the visualizer through a databinding. In some cases, the visualizer may collect user input that maybe processed by a remote device on which the data browser may operate.

Force Directed Graph as Input Mechanism for Tracer

A tracer may use a force directed graph as an input mechanism. The forcedirected graph may allow a user to select and manipulate nodes and edgesof the graph, which may represent various elements of an application.Once selected, the user may be able to apply various actions to theelements, such as causing additional tracing to be applied to theelements or to related elements.

A force directed graph or other visualization may present applicationelements in different groupings or presentations, which may help a usersee relationships within the elements. By using a force directed graphor other visualization as an input to the tracer, a user may be able toeasily select elements and related elements that would otherwise bedifficult to select.

The graph may contain a legend that may show groups of elements. Thelegend may include hot spots or other user interface controls with whicha user may select a subset of the elements.

The user interface may include an additional menu of options that mayuse the selected elements as input. The additional menu may includevarious actions that may be taken by the tracer supplying the displayeddata. A configuration file may be updated and sent to the tracer tochange the tracer behavior.

Transformation Definition for Trace Data

Trace data may be prepared for display by applying predefined oruser-defined transformations. A visualization of the data may include auser interface through which a user may select one or more predefinedtransformations or enter executable code or expressions that may createa new transformation.

The user-entered expression may define changes that may be applied todata in preparation for visualization. The changes may performstatistical analysis, apply arithmetic functions, combine data fields,merge external data, or other functions. The expressions may allow auser to create transformations that address specific scenarios that maynot be envisioned when a visualization may be created.

The expression may be inserted into a data processing pipeline for adata feed. In some cases, the data processing pipeline may be a realtime pipeline that may receive, process, and display real time data.

Throughout this specification and claims, the terms “profiler”,“tracer”, and “instrumentation” are used interchangeably. These termsrefer to any mechanism that may collect data when an application isexecuted. In a classic definition, “instrumentation” may refer to stubs,hooks, or other data collection mechanisms that may be inserted intoexecutable code and thereby change the executable code, whereas“profiler” or “tracer” may classically refer to data collectionmechanisms that may not change the executable code. The use of any ofthese terms and their derivatives may implicate or imply the other. Forexample, data collection using a “tracer” may be performed usingnon-contact data collection in the classic sense of a “tracer” as wellas data collection using the classic definition of “instrumentation”where the executable code may be changed. Similarly, data collectedthrough “instrumentation” may include data collection using non-contactdata collection mechanisms.

Further, data collected through “profiling”, “tracing”, and“instrumentation” may include any type of data that may be collected,including performance related data such as processing times, throughput,performance counters, and the like. The collected data may includefunction names, parameters passed, memory object names and contents,messages passed, message contents, registry settings, register contents,error flags, interrupts, or any other parameter or other collectabledata regarding an application being traced.

Throughout this specification and claims, the term “executionenvironment” may be used to refer to any type of supporting softwareused to execute an application. An example of an execution environmentis an operating system. In some illustrations, an “executionenvironment” may be shown separately from an operating system. This maybe to illustrate a virtual machine, such as a process virtual machine,that provides various support functions for an application. In otherembodiments, a virtual machine may be a system virtual machine that mayinclude its own internal operating system and may simulate an entirecomputer system. Throughout this specification and claims, the term“execution environment” includes operating systems and other systemsthat may or may not have readily identifiable “virtual machines” orother supporting software.

Throughout this specification, like reference numbers signify the sameelements throughout the description of the figures.

When elements are referred to as being “connected” or “coupled,” theelements can be directly connected or coupled together or one or moreintervening elements may also be present. In contrast, when elements arereferred to as being “directly connected” or “directly coupled,” thereare no intervening elements present.

The subject matter may be embodied as devices, systems, methods, and/orcomputer program products. Accordingly, some or all of the subjectmatter may be embodied in hardware and/or in software (includingfirmware, resident software, micro-code, state machines, gate arrays,etc.) Furthermore, the subject matter may take the form of a computerprogram product on a computer-usable or computer-readable storage mediumhaving computer-usable or computer-readable program code embodied in themedium for use by or in connection with an instruction execution system.In the context of this document, a computer-usable or computer-readablemedium may be any medium that can contain, store, communicate,propagate, or transport the program for use by or in connection with theinstruction execution system, apparatus, or device.

The computer-usable or computer-readable medium may be, for example butnot limited to, an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system, apparatus, device, or propagationmedium. By way of example, and not limitation, computer readable mediamay comprise computer storage media and communication media.

Computer storage media includes volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules or other data. Computer storage media includes, but isnot limited to, RAM, ROM, EEPROM, flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tostore the desired information and which can accessed by an instructionexecution system. Note that the computer-usable or computer-readablemedium could be paper or another suitable medium upon which the programis printed, as the program can be electronically captured, via, forinstance, optical scanning of the paper or other medium, then compiled,interpreted, of otherwise processed in a suitable manner, if necessary,and then stored in a computer memory.

When the subject matter is embodied in the general context ofcomputer-executable instructions, the embodiment may comprise programmodules, executed by one or more systems, computers, or other devices.Generally, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular abstract data types. Typically, the functionalityof the program modules may be combined or distributed as desired invarious embodiments.

FIG. 1 is a diagram of an embodiment 100 showing an example forcedirected graph. Embodiment 100 is an example of a force directed graphthat may show objects in a message passing relationship with each other,as well as various controls that may be used to view the graph with asequence of data sets.

The force directed graph of embodiment 100 may illustrate messagespassed within a message passing environment. As an example of such anenvironment, independent compute elements may process portions of anapplication. During processing, each compute element may pass messagesto another compute element that contain data, instructions, or otherelements. In such environments, a force direct graph may be used tovisualize the computational elements and the activity between theelements. In many cases, force directed graphs may be used to identifybottlenecks or other irregularities during execution.

The force directed graph of embodiment 100 may illustrate one timeperiod during the execution of an application. In such embodiments, theexecution of an application may be traced over time and the forcedirected graph may illustrate how the application behaves. The forcedirected graph of embodiment 100 may be updated periodically with newlycollected data, which may visually show operations of the application.

The force directed graph of embodiment 100 may illustrate the operationsof an application. In some embodiments, the force directed graph mayillustrate the system state of a device or application at discreteperiods of time. An example of a graph illustrating the system state mayinclude nodes representing the state of memory objects, functions,input/output devices, memory storage devices, or other hardware orsoftware objects. In some embodiments, the force directed graph mayillustrate activities that may occur between two periods of time. Anexample of such a graph may include functions or processes and messagespassed between processes.

The force directed graph may show nodes 102, 104, 106 and 108 connectedby various edges. Edge 112 connects nodes 102 and 104. Edge 114 connectsnodes 102 and 106, while edge 116 connects nodes 102 and 108 and edge118 connects nodes 106 and 104. Additional nodes and edges are alsoillustrated.

A force directed graph may be computed by applying an attractive forceconnecting two nodes with an edge, and at the same time applying arepulsive force to nodes in general. In many embodiments, a forcedirected graph may be displayed in an interactive manner such that auser may be able to perturb the graph by clicking and dragging an objector through some other mechanism. As a perturbation is introduced, aninteractive graph may show the various nodes and edges change position.

The force directed graph of embodiment 100 may show additional dataelements. For example, the relative size, shape, and color of thevarious nodes may be configured to indicate different characteristics ofthe node. In another example, the edges may display characteristicsusing thickness, color, and other visual elements.

When a force directed graph displays the execution of an application,the nodes may represent compute elements. The compute elements may beprocesses, threads, processors, devices, or other elements that may passmessages to other elements. In such a graph, the edges may representmessages passed between compute elements.

Nodes representing compute elements may be modified to reflectadditional data. For example, the color or shape of the node may bemodified to show groupings of the compute elements. In the example ofembodiment 100, a legend 126 illustrates different colors or patternsapplied to the nodes and the meaning of the patterns. Nodes representingcompute elements from library A 120 may include nodes 102 and 108. Nodesrepresenting compute elements from library B 122 may include nodes 104and 106. Node 110 may represent a core process 124.

Groupings may reflect different shared characteristics of the objects.For example, nodes may be grouped by library, code module, or othergroup, and such a grouping may assist a developer in understandingprogram flow. In another example, nodes may be grouped by memoryconsumption, where those nodes representing compute elements thatconsume large amounts of data are grouped together, or where computeelements that reference specific groups of memory objects are groupedtogether. In another example, processes or functions that operate on aspecific process scheduler may be shown as groups. In still anotherexample, nodes that may be related to a memory domain may be grouped.

In some embodiments, a legend may be shown as part of a graph. Thelegend may have colors, shapes, or other visual elements and acorresponding label. In some embodiments, the legend may have aselection mechanism whereby a user may be able to select a groupingusing a drop down menu or other selection tool. In some suchembodiments, a user may be able to select one visual effect tocorrespond to one grouping while another visual effect may correspond toanother grouping. For example, a legend may be used to configuregrouping by memory domain to be illustrated by shapes that representeach domain, while nodes relating to specific services may be grouped bycolor.

The legend 126 may have a selection tool for selecting a grouping to beshown. A toggle button 136 may open a drop down list that may containseveral options. In the case of embodiment 100, the options may includegrouping by processor 138, memory domain 140, scheduler 142, and service144. The service 144 selection is currently selected, as indicated by astar. When a user selects a different grouping, the grouping may beapplied to the various nodes by changing the color, shape, or othervisual element.

The size of the various nodes may reflect different aspects of thecomputational elements. For example, the size may represent the amountof computation performed by a particular element, the number of timesthe element was called, the amount of data handled by the element, orother factors. In some cases, a specific color may be applied to anelement that receives input data from an external source and a differentcolor may be applied to an element that transmits output data.

Likewise, the edges may be modified to show various aspects of themessages. For example, the messages may be aggregated to show the numberof messages along a specific path, the frequency of messages, the datapayloads of the messages, as well as directionality of the messages andother features. The edges corresponding to the messages may be modifiedusing different thicknesses, colors, or other visual elements toillustrate one or more of the aggregated parameters.

The operation of an application may produce many messages that may bepassed over time. Such time-related data may be displayed using a timeseries of datasets, where each dataset may reflect the state of theapplication at a period of time or as an aggregation of the messagespassed during a time interval.

In some embodiments, a tracing system may collect message passinginformation from an active application and store the collected data in adatabase. An aggregator may analyze the database to summarize messagepassing activities for individual time intervals. In some cases, suchsummarizing may be performed by the tracing system without storingmessage passing data in a separate database.

Aggregated data may be displayed in a force directed graph by updatingthe data within the force directed graph. In many visualizations of aforce directed graph, the dataset may be updated, causing the forcedirected graph to reposition itself with the updated data.

A force directed graph may reflect the operations of an application inreal time. In such an embodiment, a tracer system may collect messagepassing data from a compute environment and aggregate the data forpresentation. The data may be updated at a periodic interval, such asevery second, then transmitted to a system displaying the force directedgraph. The force directed graph may be updated and change with eachupdate, allowing a user to visualize the operations of the applicationin real time or near real time.

When datasets may be collected and stored in such an embodiment, acontrol panel user interface may allow a user to browse and view thevarious datasets. For example, a reverse button 128 may cause older datasets to be shown in reverse order. A play button 130 and a pause button132 may start and stop a force directed graph to be updated. A fastforward button 134 may cause the playback to occur at a faster thannormal speed.

FIG. 2 is a diagram of an embodiment 200 showing a computing environmentthat may collect and display message passing data in a graph. Embodiment200 illustrates hardware components that may deliver the operationsdescribed in embodiment 100, as well as other embodiments.

The diagram of FIG. 2 illustrates functional components of a system. Insome cases, the component may be a hardware component, a softwarecomponent, or a combination of hardware and software. Some of thecomponents may be application level software, while other components maybe execution environment level components. In some cases, the connectionof one component to another may be a close connection where two or morecomponents are operating on a single hardware platform. In other cases,the connections may be made over network connections spanning longdistances. Each embodiment may use different hardware, software, andinterconnection architectures to achieve the functions described.

Embodiment 200 illustrates a device 202 that may have a hardwareplatform 204 and various software components. The device 202 asillustrated represents a conventional computing device, although otherembodiments may have different configurations, architectures, orcomponents.

In many embodiments, the optimization server 202 may be a servercomputer. In some embodiments, the optimization server 202 may stillalso be a desktop computer, laptop computer, netbook computer, tablet orslate computer, wireless handset, cellular telephone, game console orany other type of computing device. In some cases, the optimizationserver 202 may be deployed on a computing cluster, cloud computingenvironment, or other hardware platform.

The hardware platform 204 may include a processor 208, random accessmemory 210, and nonvolatile storage 212. The hardware platform 204 mayalso include a user interface 214 and network interface 216.

The random access memory 210 may be storage that contains data objectsand executable code that can be quickly accessed by the processors 208.In many embodiments, the random access memory 210 may have a high-speedbus connecting the memory 210 to the processors 208.

The nonvolatile storage 212 may be storage that persists after thedevice 202 is shut down. The nonvolatile storage 212 may be any type ofstorage device, including hard disk, solid state memory devices,magnetic tape, optical storage, or other type of storage. Thenonvolatile storage 212 may be read only or read/write capable. In someembodiments, the nonvolatile storage 212 may be cloud based, networkstorage, or other storage that may be accessed over a networkconnection.

The user interface 214 may be any type of hardware capable of displayingoutput and receiving input from a user. In many cases, the outputdisplay may be a graphical display monitor, although output devices mayinclude lights and other visual output, audio output, kinetic actuatoroutput, as well as other output devices. Conventional input devices mayinclude keyboards and pointing devices such as a mouse, stylus,trackball, or other pointing device. Other input devices may includevarious sensors, including biometric input devices, audio and videoinput devices, and other sensors.

The network interface 216 may be any type of connection to anothercomputer. In many embodiments, the network interface 216 may be a wiredEthernet connection. Other embodiments may include wired or wirelessconnections over various communication protocols.

The software components 206 may include an operating system 218 on whichvarious applications 244 and services may operate. An operating systemmay provide an abstraction layer between executing routines and thehardware components 204, and may include various routines and functionsthat communicate directly with various hardware components.

Each of the various devices illustrated in embodiment 200 may have ahardware platform. The respective hardware platforms may be similar tothe hardware platform 204. The devices may be any type of hardwareplatform, such as a personal computer, server computer, game console,tablet computer, mobile telephone, or any other device with aprogrammable processor.

The analyzer device 202 may contain an operating system 218, which maysupport various other software components. The components may include ananalyzer 220, which may prepare data for visualization. The analyzer 220may take data collected while an application runs using an extractor 222and aggregate the data using an aggregator 224 to create data that maybe visualized by a visualizer 226.

A collector system 230 may operate on a hardware platform 232 and have acollector 234 that may gather trace data collected while an applicationexecutes and store the data in a database 236. These data may then beprocessed by the analyzer 220.

A client device 238 may have a hardware platform 240 in which a browser242 may execute. The browser 242 may display a graph 244 that may begenerated by the visualizer 226.

The architecture of embodiment 200 illustrates a system where ananalyzer 202 may prepare data for a visualizer 226 to display a graph244 that may be rendered in a browser 242. In such an architecture,message passing data may be collected on an ongoing basis, then aseparate processing step may be performed by the analyzer 220. Such anarchitecture may allow multiple analyses of the raw data to beperformed.

For example, when the raw data are stored prior to analysis, time seriesof datasets may be configured with different periods. For example, atime series for a long time period may be created that illustrateschanges that may occur over a long period of time. At the same time, adetailed time series may be created for very small time periods. Alonger time period may help a user understand long term activities thatoccur in an application, while the detailed time series may show a muchhigher level of detail for debugging, for example.

Another embodiment may include some of the operations of the collector234 and analyzer 220 into a single component. In such embodiments, thedata may be analyzed, aggregated, and prepared for viewing in a singlesoftware component. Such a component may be integrated into a tracerthat runs on the same device as the application under test in somecases. Still other architectures may perform similar operations but areconfigured differently.

An example of a compute environment 246 illustrates multiple deviceswhich may interact in a high performance computing environment or otherenvironment where message passing may be deployed. Each device 248, 256,264, and 272 may have a respective hardware platform 250, 258, 266, and274. An application 252, 260, 268, and 272 may execute with a respectivetracer 254, 262, 270, and 278 on the respective hardware platforms.

The example of compute environment 246 may be deployed in a clusterenvironment, dispersed computing environment, or some other manner suchthat the various devices may communicate with each other. Theapplications may contain the same or different executable code that maybe configured to pass messages to other devices in order to execute aworkload that may be larger than can be performed on a single device.

Another example of a compute environment may be an application device280 that may have a hardware platform 282 which may contain one or moreprocessors. On each processor, multiple processes may execute and passmessages between the processes. In the example of device 280, fourprocessors 284, 288, 292, and 296 are illustrated as executing processes286, 290, 294, and 298.

One example of such a system may deploy a functional language, such asErlang, whereby a single application may be executed using manyindividual processes, threads, or other compute elements. The variouselements may communicate with each other by passing messages within thedevice 280. In some applications, many thousands, tens of thousands, oreven millions of processes and messages may make up an applicationduring execution.

FIG. 3 is a flowchart illustration of an embodiment 300 showing a methodfor monitoring data. Embodiment 300 illustrates the operations of atracer that may gather message passing data and store the data in adatabase for later analysis. The operations of embodiment 300 mayreflect the operations of tracer 254, for example, in embodiment 200.

Other embodiments may use different sequencing, additional or fewersteps, and different nomenclature or terminology to accomplish similarfunctions. In some embodiments, various operations or set of operationsmay be performed in parallel with other operations, either in asynchronous or asynchronous manner. The steps selected here were chosento illustrate some principles of operations in a simplified form.

Embodiment 300 may illustrate a method whereby each message passed maybe analyzed to gather various data. The architecture of embodiment 300illustrates one routine that may monitor an application and when amessage is identified, a data gatherer instance may be deployed. Thedata gatherer instance may collect various data and store the data.

An application may be started in block 302 and monitoring may start inblock 304. When a message is identified in block 306, a data gathererinstance 312 may be deployed. The monitoring may continue in block 308until another message is identified, causing the process to return toblock 306 and launch another data gatherer instance 312. When no moremessages are identified in block 308, the process may end in block 310.

The data gatherer instance 312 may reflect the operations of a processor function that may operate on a single message. From the message, thesender and receiver may be identified in block 314. The data transmittedin the message payload may be gathered in block 316.

Information about the sender may be gathered in block 318 andinformation about the receiver may be gathered in block 320. Suchinformation may include how much processing may be performed, the natureof the processing, or other information. Once all of the information hasbeen gathered, the message data may be stored in block 322.

FIG. 4 is a flowchart illustration of an embodiment 400 showing a methodfor analyzing and aggregating data. Embodiment 400 illustrates theoperations of an analyzer that may analyzed and aggregate data collectedin embodiment 300. The operations of embodiment 400 may reflect theoperations of analyzer 220, for example, in embodiment 200.

Other embodiments may use different sequencing, additional or fewersteps, and different nomenclature or terminology to accomplish similarfunctions. In some embodiments, various operations or set of operationsmay be performed in parallel with other operations, either in asynchronous or asynchronous manner. The steps selected here were chosento illustrate some principles of operations in a simplified form.

Embodiment 400 illustrates an example of how message passing data may beanalyzed prior to visualization. In some embodiments, the operations ofembodiment 400 may be performed inline with the operations of a datamonitoring method. Some such embodiments may apply the aggregationmethod 400 within a tracer to create data that may be ready for displayas quickly as possible, so as to enable real time or near-real timevisualizations of an application.

The periodicity of a dataset may be determined in block 402. Theperiodicity may define the time interval of a time series. Formonitoring an application, the periodicity of a time series may bevalues less than a millisecond, in the sub-second range, in the singledigit seconds, or longer. Depending on the application, some instancesmay have periods of tens of seconds, single digit minutes, tens ofminutes, hours, days, weeks, or longer.

A starting period may be selected in block 404. Messages passed withinthe period may be identified in block 406. In some cases, the selectedmessages may have multiple messages that communicate between computeelements, which may be sorted by the message path in block 408.

For each message path in block 410, a summary of the messages passed maybe made in block 412. The summary may include the number of messages,direction of those messages, amount of data passed, frequency, or otherstatistics. In some cases, the summaries may be nonlinear summaries. Forexample, a logarithm, square, cubic, or other function may be used togenerate aggregated summaries. In many data collection scenarios, someobjects may be accessed one, two, or a handful of times while otherobjects may be accessed thousands or even millions of time. In order topresent such data comparisons within a graph, a nonlinear scaling of thedata may be used.

Each node may be identified in block 414. For each node in block 416,the node activity may be summarized in block 418. The summary mayinclude the amount of computation performed by the compute element,input or output data passed to or from the element, type of computingperformed, as well as statistics relating to the computation such as thetime busy, waiting, performing garbage collection, heap size, memorycalls, or other information.

After analyzing all of the message data for the period of time, themessage data may be stored in block 420 as a data set. If another periodis to be analyzed in block 422, the period may be incremented in block424 and the process may return to block 404. When no more periods are tobe analyzed in block 422, the process may end in block 426.

FIG. 5 is a flowchart illustration of an embodiment 500 showing a methodfor deploying and updating a graph. Embodiment 500 illustrates theoperations of a visualizer of the data aggregated in embodiment 400. Theoperations of embodiment 500 may reflect the operations of visualizer226, for example, in embodiment 200.

Other embodiments may use different sequencing, additional or fewersteps, and different nomenclature or terminology to accomplish similarfunctions. In some embodiments, various operations or set of operationsmay be performed in parallel with other operations, either in asynchronous or asynchronous manner. The steps selected here were chosento illustrate some principles of operations in a simplified form.

Embodiment 500 illustrates two separate activities that may be performedto display a graph. A baseline graph may be created and displayed inblock 502 and highlights may be added in block 504 on an ongoing basis.

The baseline graph of block 502 may display a graph that contains allnodes and edges from a large database. In many instances, such a graphmay represent the long term operations of an application and may beuseful to understand the application.

A database may be selected in block 506 to visualize. All of the timeperiods may be analyzed in block 508 to identify all nodes in block 510and all message paths in block 512. In some embodiments, summarystatistics may be generated over all of the nodes and edges in blocks510 and 512, respectively. The corresponding graph may be generated inblock 514.

The baseline graph generated in block 514 may be a static graph thatillustrates summary statistics from many time periods. In manyembodiments, the baseline graph may serve as a framework for otherillustrations.

For example, the operations of highlighting activity in block 504 mayidentify changes to the graph from a specific time period and overlaythose changes on the baseline graph. In one such example, a baselinegraph may contain representations of all the computational elements andmessages that may be passed during the lifetime of an application. Inorder to see the recent operations, operations in a current time periodmay be identified and displayed with visual highlighting, where othercompute elements and message paths that were not exercised in the timeperiod may be displayed without highlighting. In such an example, all ofthe nodes and edges may be displayed in a greyed-out fashion withcurrently executing nodes and messages shown in a vibrant color.

In such a display, the baseline graph may provide a visual context onwhich the current changes may be displayed.

The operations for highlighting activities in block 504 may includereceiving user input that selects a time period in block 518. A datasetfor the selected time period may be retrieved in block 520.

Nodes and edges within the selected time period may be identified inblock 522 and displayed as highlighted in block 524. Those nodes andedges not changed in the time period may be identified in block 526 anddisplayed as not highlighted in block 528.

In many cases, the user may select a current time period to display.Such a selection may update a graph in real time or near-real time. Whenan embodiment incorporates various navigation tools, such as the controlbuttons of embodiment 100, a user may be able to browse, scroll, or usesome other mechanism to identify a data set to display.

When a visualization is updated over period of time while an applicationexecutes, some embodiments may display only those elements that havebeen changed in the last sampling period of the time series. In suchembodiments, the shape of a force directed graph or other visualizationmay change rapidly, especially when the time period may be very short.

Some such embodiments may decay and remove elements over multipleupdates. In such an embodiment, each node or edge may be displayed for apredefined number of periods, then removed from the graph. As the nodebecomes older and is not used, the node may be displayed in a greyed-outfashion in some such embodiments.

FIG. 6 is a flowchart illustration of an embodiment 600 showing a methodfor generating traces of an object within a graph. Embodiment 600illustrates another version of changes that may be made to a baselinegraph, similar to the operations of embodiment 500. In some cases, theoperations of embodiment 600 may be modified to update a graph withoutusing the baseline graph.

Other embodiments may use different sequencing, additional or fewersteps, and different nomenclature or terminology to accomplish similarfunctions. In some embodiments, various operations or set of operationsmay be performed in parallel with other operations, either in asynchronous or asynchronous manner. The steps selected here were chosento illustrate some principles of operations in a simplified form.

The operations of embodiment 600 may illustrate one method by which anobject and its effects may be illustrated within a graph, such as aforce directed graph. After defining an object, one, two, or moregenerations of messages stemming from the object may be highlighted tocreate an update to a baseline graph. The update may be displayed on abaseline graph as highlighted nodes and edges to illustrate the effectsof the selected object.

A baseline graph may be created and displayed in block 602. An exampleof the operations of block 602 may be found in block 502 of embodiment500.

A selection for a trace object may be received in block 604. A traceobject may be any event, memory object, condition, function, or otherparameter that a user may wish to examine. A user may be able to definea memory object to trace, for example, when the memory object is changedor is set to a specific value. Once such a condition is met, the effectsof the condition may be illustrated.

After defining a trace object, instances of the trace object may besearched in the message passing database. In some cases, multipleinstances of the condition may be identified. When multiple instancesexist, a time series of datasets may be generated for each instance anda user may be able to select between the instances to view the timeseries datasets.

For each instance in block 608, a starting point for the sequence may beidentified. The starting point may be a starting time or period that thecondition originates.

Any messages referring to the trace object may be identified in block612 and added to a trace list. A message may refer to a trace objectwhen the selected object interacts with a compute element and thecompute element passes a message to another compute element. Forexample, a trace object may be an input event that may be processed by afirst compute element, which may send a message to two other computeelements. Such messages may be added to a trace list in block 612.

The operations of block 612 may identify an original set of messagesthat may be triggered by a trace object or condition. For each messagein the trace list in block 614, downstream messages may be identified inblock 616 and added to the trace list. The downstream messages may bemessages that may have been triggered by the original messagesidentified in block 612. If additional generations of messages aredesired in block 618, the process may return to block 614 to addadditional messages.

When all of the desired generations of messages may be identified inblock 618, a time series of all the generations of messages may becreated in block 620. The time series may include separate data setsthat represent individual generations of messages that may be passedfrom compute element to compute element in response to the tracecondition.

An instance may be selected in block 622 and the messages may bedisplayed as highlighted messages in block 624. In many cases, thehighlighted messages may be displayed on the framework of a baselinegraph that may be created and displayed in block 602.

FIG. 7 is a diagram illustration of an embodiment 700 showing a timeseries of data sets displayed as a force directed graph. Embodiment 700shows a time A 702, time B 704, time C 706, and time D 708. Embodiment700 is a simplified example of a force directed graph that may grow anddecay over time.

Embodiment 700 illustrates a simple force directed graph that may becreated and may grow and decay with each successive time step.

At the initial time step, time A 702, nodes 710, 712, and 714 areillustrated. At the second time step, time B 704, node 716 may be added.

In the third time step, time C 706, nodes 718 are added while node 712may be either removed or displayed in a greyed out mode. In the fourthtime step, time D 708, nodes 720 are added and nodes 710 and 712 may beremoved or displayed in a greyed out mode.

Embodiment 700 shows the progression of a trace of an application overtime. In each time period, the compute elements may be represented bythe nodes and messages passed between the compute elements may berepresented by the edges of the graph. Initially, three compute elementsare present and two message paths were exercised in time A 702. As timeprogresses, additional compute elements are used and additional messagepaths are exercised.

At time C 706, one of the nodes and message paths may no longer be used.In such a case, some embodiments may preserve the representation of node712 as a greyed-out version. Other embodiments may remove the node 712when the node 712 has not been exercised.

Some embodiments may decay the representations over time. In suchembodiments, each node may be illustrated for several time periods, evenwhen the node is not exercised in the successive time periods. The nodemay be illustrated with full color intensity when it is initiallydisplayed, then the node may be illustrated with less intensity at eachsuccessive time period until the point where the node may be removedfrom the graph. Such an embodiment may keep a node visible for severaltime periods so that a user may visualize the node, but may remove thenode when the node has not been exercised.

In an example, a node may be displayed with full intensity for two,three, or more time periods in an animated representation. After theinitial display, the node may be illustrated with decreasing intensityfor another 15 time periods, after which the node may be removed fromthe graph. In some embodiments, the time period for decay and for theinitial representation may be adjustable by a user.

FIGS. 8A, 8B, and 8C are diagram illustrations of example embodiments802, 804, and 806 showing force directed graphs in a user interface.Embodiments 802, 804, and 806 illustrate a sequence of interactions thatmay be performed with user input to select a group of graphicalelements, then perform an action on the selected elements. Each of theembodiments 802, 804, and 806 comprises a force directed graph and alegend 808.

Embodiments 802, 804, and 806 may illustrate one mechanism to selectmultiple elements from a force directed graph: such a mechanism may bean area selection using a rectangular window. Other embodiments maypermit a user to select groups of elements by other selectionmechanisms, such as a lasso tool, clicking on a succession of elements,or other mechanisms.

The nodes of embodiments 802, 804, and 806 are commonly labeled. Theforce directed graphs are composed of nodes 810, 812, 814, 816, 818, and820.

Embodiment 802 may represent a force directed graph as displayed in auser interface. Embodiment 804 may illustrate the force directed graphof embodiment 802 with a window selection. The window selection may bedefined by points 822 and 824 to define a selection box 826.

The selection box 826 may be created by a user by defining the points822 and 824. One mechanism for creating the points 822 and 824 may be toclick and drag a stylus, cursor, or other pointing tool within thedisplayed area of the force directed graph.

The selection box 826 may capture node 812 and the group of nodes 814,which may illustrate one use scenario. Specifically, a force directedgraph or other visualization may illustrate relationships and groups ofelements in ways that may not be readily apparent without thevisualization. The selection mechanism performed with the visualizationmay allow a user to select related objects quickly and easily,especially when the relationships may not be apparent by othermechanisms.

For example, node 812 may represent one function in a library and nodes814 may represent a group of functions in a second library. Wheninitially started, a programmer may or may not be able to determine thatthe two sets of functions were related. After running a tracer andvisualizing the relationships in a force directed graph 802, theprogrammer may be able to identify the relationships.

After selecting node 812 and group of nodes 814, the user may performadditional operations, as illustrated in embodiment 806.

In embodiment 806, the selected items 812 and 814 may be illustrated ashighlighted while the remaining portions of the force directed graph maybe illustrated as not highlighted. Some embodiments may display nonhighlighted elements using transparency, color schemes such asgreyed-out colors, or other visual cues. Some embodiments may displayhighlighted elements using brighter or more vibrant colors, differentcolor pallets, boldness, or other visual cues.

Once selected, the items may be have some activity or changes to beapplied to the selected group. Such a change may be selected from a userinterface component 828, which may have various options 830 and 832.

In some embodiments, the selected activity may cause the tracer tochange the way data are collected. In such embodiments, the forcedirected graph may be a user interface component for controlling ormanaging a tracer. An example of such a change may to increase thedetail of tracing for the selected elements. Such a change may increasethe tracing data for subsequent time slices. In another example, thetracer may be instructed to remove the selected elements from futuredata sets. In such a change, the tracer may reduce the amount of datacollected in future time slices.

In some embodiments, the selected activity may cause a preprocessor tochange the way trace data are processed or presented on the userinterface. An example may be to show cumulative data for the selectedelements or to visually highlight objects that may flow from theselected elements. Such selections may not cause the tracer to changethe data collected but may cause a preprocessor or visualizer to changethe way the data are illustrated.

FIGS. 9A, 9B, and 9C are diagram illustrations of example embodiments902, 904, and 906 showing force directed graphs in a user interface.Embodiments 902, 904, and 906 illustrate a sequence of interactions thatmay be performed with user input to select a group of graphicalelements, then perform an action on the selected elements. Each of theembodiments 902, 904, and 906 comprises a force directed graph and alegend 908.

Embodiments 902, 904, and 906 may illustrate one mechanism to selectmultiple elements from a force directed graph: such a mechanism may usea legend label to select members of a group of elements.

The nodes of embodiments 902, 904, and 906 are commonly labeled. Theforce directed graphs are composed of nodes 910, 912, 914, 916, 918, and920.

Embodiment 902 may represent a force directed graph as displayed in auser interface. Embodiment 904 may illustrate the force directed graphof embodiment 902 with a selection made from the legend 908. Theselection 922 within the legend 908 may cause all of the objects withmembership in group B to be selected and highlighted.

Embodiment 904 illustrates nodes 910 and 920 as the selected members ofgroup B, while the remaining elements may be illustrated as nothighlighted. The relationships of nodes 910 and 920 are also illustratedas highlighted, while the remaining relationships or edges may beillustrated as not highlighted.

Once the elements associated with the selection 922 are selected, a userinterface component 924 may be presented. A user may be able to selectbetween options 926 and 928 to apply changes to a tracer or changes tohow the data are preprocessed and displayed, in a similar manner as withthe user interface component 828. When the selections may be made, alaunch button 930 may be used to cause the changes to be implemented.

FIG. 10 is a flowchart illustration of an embodiment 1000 showing amethod for controlling a tracer through user interactions with a graph.Embodiment 1000 illustrates a simplified method that may be performedwith the user interface examples of embodiments 802, 804, and 806 aswell as embodiment 902, 904, and 906.

Other embodiments may use different sequencing, additional or fewersteps, and different nomenclature or terminology to accomplish similarfunctions. In some embodiments, various operations or set of operationsmay be performed in parallel with other operations, either in asynchronous or asynchronous manner. The steps selected here were chosento illustrate some principles of operations in a simplified form.

An initial data set may be received in block 1002, and a graph may bedisplayed in block 1004. If there is no user selection in block 106, anew data set may be received in block 1008 and the process may return toblock 1004 to show an updated graph.

The loop of blocks 1004 through 1008 may illustrate a normal operationof a user interface display for time series data. The graph may becontinually updates with the sequence of data sets within the timeseries.

In block 1006, a user may select one or more elements of the graph. Insome embodiments, updating may be paused in block 1010. The updating maybe paused in cases where the graph may change rapidly and the user maynot be able to select a set of desired elements while the graph changes.

Once the elements may be selected, changes to be performed on thoseitems may be selected in block 1012. The changes may be transmitted tothe tracer in block 1014 and the process may return to block 1008, wherea new data set may be received.

FIG. 11 is a diagram of an embodiment 1100 showing a computingenvironment that may collect and display message passing data in agraph, then use the graph to control how those data are collected.Embodiment 1100 illustrates hardware components that may deliver theoperations described in embodiment 1000, as well as other embodiments.

The diagram of FIG. 11 illustrates functional components of a system. Insome cases, the component may be a hardware component, a softwarecomponent, or a combination of hardware and software. Some of thecomponents may be application level software, while other components maybe execution environment level components. In some cases, the connectionof one component to another may be a close connection where two or morecomponents are operating on a single hardware platform. In other cases,the connections may be made over network connections spanning longdistances. Each embodiment may use different hardware, software, andinterconnection architectures to achieve the functions described.

Embodiment 1100 illustrates a network environment in whichvisualizations of trace data may be used to control the traceroperation. A visualization device 1102 may display a graph and provide auser interface, while a controller device 1104 may control the data setsto be displayed, as well as manage the operations of a tracer. Thecontroller device 1104 may retrieve data from a data repository 1106. Atracer device 1108 may collect trace data while running an application1138. All of the various devices may be connected with a network 1110.

The visualizer device 1102 may include on a hardware platform 1112, onwhich a browser 1114 may execute. A visualizer 1116 may be code runningin the browser 1114 that may generate a graph within the user interface1118.

The controller device 1104 may operate as a backend server that performsseveral services that support the operations of the visualizer device1102.

The example of embodiment 1100 illustrates an architecture where thevisualizer 1116 may reside on a client device, while other services mayreside on a server device. The visualizer 1116 may be located on theclient device to improve the user experience with an animated graph.When the visualizer 1116 is located on a user's device, the smoothnessof animation and responsiveness of the graph may be improved overarchitectures where rendering and visualization may be performed onremote devices.

Embodiments where remote devices perform some or all of thevisualization may be useful in situations where a client device may nothave sufficient processing power to render a graph. Such embodiments mayenable more complex and detailed renderings than may be generated withclient-side visualization tools.

The controller device 1104 may provide two different functions, one ofwhich may be as a data browser 1126 through which data for avisualization may be retrieved and prepared, as well as a tracerconfiguration manager 1122, where changes to a tracer may be created anddispatched. A user interface communicator 1124 may be accessed throughcomponents in the browser 1114 to cause changes in the data browser 1126or the tracer configuration manager 1122.

In some embodiments, a user interface 1118 may include a dialog box,selection tool, or other user interface component that may be used toconfigure or change configuration of a tracer. Such configuration mayinclude items relating to the general operation of the tracer, such assampling frequency, resources allocated to the tracer, conditions forstarting or stopping the tracer, and other general operational options.In some embodiments, such changes may be applied generally or to itemsselected from the graph.

The data browser 1126 may retrieve data sets 1128 from the datarepository 1106 and prepare the data sets for viewing by the visualizer1116. The data browser 1126 may be responsive to playback controls, suchas the controls 128 through 134 in embodiment 100.

The data browser 1126 in normal playback mode may retrieve data sets1128 and make the data sets available to the visualizer 1116. In manyembodiments, such an action may be performed on a recurring, periodicbasis according to the time series represented by the data sets 1128.For example, a time series may be created where data sets 1128 mayrepresent each second of time during a time series. In such an example,the data browser 1126 may make each successive data set available eachsecond.

The user interface communicator 1124 may receive commands from thebrowser 1114 to pause, rewind, fast forward, play, stop, and othercommands. These commands may be passed to the data browser 1126 whichmay begin retrieving data sets 1128 and presenting the data sets in therequested sequence and in the requested frame rate or speed.

The tracer configuration manager 1122 may receive inputs from the userinterface communicator 1124, where the inputs may define changes to bemade to trace data. The changes may reflect additional data points thatmay be collected, as well as data points that may be removed or otherchanges. In some cases, the changes may reflect the behavior oroperational changes, such as when the tracer may be executed, thefrequency of data collection, or other changes.

The tracer device 1108 may operate on a hardware platform 1130 and havean instrumented execution environment 1132 that may include a tracer1134 and a configuration 1136 for the tracer 1134. The tracerconfiguration manager 1122 may update the configuration 1136 to causethe tracer 1134 to change behavior.

An application 1138 may execute in the instrumented executionenvironment 1132, allowing the tracer 1134 to generate trace data. Thetrace data may be transmitted to the data repository 1106 by a datatransmitter 1140. The data transmitter 1140 may periodically communicatewith the data repository 1106 to transmit any collected data from thetracer 1134.

FIG. 12 is a diagram illustration of an embodiment 1200 showing aprocess for visualizing data from a tracer. Embodiment 1200 mayillustrate a processing pipeline where transformations may be inserted.In some embodiments, user written executable code may be inserted intothe processing pipeline to prepare data for visualization in manydifferent manners.

A tracer 1202 may generate a stream of trace data that may be processedby a storage pipeline 1204. The storage pipeline 1204 may prepare andprocess the trace data using a set of transformations in block 1206 forstorage in block 1208. In some embodiments, the trace data may be acontinuous stream of data items that may be gathered by the tracer 1202.Such streams of data may increase and decrease in volume over time. Inother embodiments, the trace data may be snap shots of data reported atspecific intervals. Such streams of data may be regularly recurring.

The storage pipeline 1204 may be a set of processes that apply a set oftransformations in block 1206 to the data stream, then cause the data tobe stored in block 1208. The transformations in block 1206 may applyformatting, data analysis, aggregation, or other changes to the dataprior to storage. In many cases, the transformations in block 1206 mayperform de-duplication, compression, differencing, or other operationsthat may reduce the side of the trace data in block 1208, as well asformat the data for later retrieval.

The transformations in block 1206 may be applied prior to storage of thetrace data in block 1208. When such transformations may be lossy orotherwise diminish the accuracy, fidelity, or completeness of the data,such a transformation may be permanent in the sense that later analysismay not be able to recreate the original data.

After storage in block 1208, a visualization pipeline 1210 may apply anadditional set of transformations in block 1214 prior to visualizing thedata in block 1216. The visualization pipeline 1210 may prepare the datafor visualization. The transformations in block 1214 may not bepermanent in the sense that the raw data in block 1208 may still remain,allowing for a different set of transformations to be applied in a lateranalysis.

The transformations in block 1214 may perform various operations forpreparing data for visualizations. In some cases, the transformations inblock 1214 may perform formatting and other operations so that avisualizer in block 1216 may accept and parse the incoming data. In somecases, the transformations in block 1214 may perform filtering,aggregation, statistical analysis, and other operations that may affectwhich data are displayed and how the data are displayed.

The visualizer in block 1216 may be part of a user interface 1218through which a user may view data and control how the data aredisplayed. One mechanism for controlling how the data may be displayedmay be a user interface in block 1220 where a user may create or edittransformations. A user may also be able to store and retrieve thetransformations in block 1224 for later use. In many embodiments, alibrary or selection of several pre-configured transformations may bestored for a user to select and use with or without editing.

The user interface in block 1220 may allow a user to add and editexecutable code to define a portion of a transformation. The executablecode may be any function description, expression, or other definitionthat may be compiled, interpreted, or otherwise executed as atransformation.

Once added, a transformation may go through a malware check in block1226 before being inserted into a processing pipeline in block 1228. Atransformation may be identified to be applied prior to storage in block1206 or after storage in block 1214.

FIG. 13 is a diagram illustration of an embodiment 1300 showing anexample user interface. Embodiment 1300 may illustrate a user interfacethrough which a user may enter executable code that may be deployed as atransformation.

Embodiment 1300 may illustrate a visualization user interface 1302 thatcontains a force directed graph 1304, a legend 1306, and a control set1308. The force directed graph 1304 may display trace data in the formof nodes and edges, where the edges may represent relationships betweenobjects. The legend 1306 may show groups of elements. The control set1308 may be a set of control buttons through which a user may inputplayback commands to view different data sets in a time series of tracedata.

A window 1310 may be an interface through which a user may selectdifferent data to show in the graph. Two different options 1312 and 1314may reflect pre-defined transformations that may be selected, as well asa third option 1316 where a user may enter and edit an executableexpression in a text editor 1318. The user may also select whichprocessing pipeline to implement the transformation in the selection1320.

The transformations may cause data to be displayed, and sometimesstored, in different manners. The transformations may be defined in anexecutable language that may be compiled or interpreted to process data.In some cases, the language may enable multiple data elements to beanalyzed together. A simple example of which may be to take a differencebetween two elements.

The transformations may allow a filter to be applied, such as to showtracing data from a specific function or memory object, whileeliminating other data. In some cases, the transformations may includean expression, such as to display data from processes that operate forgreater than 10 seconds and less than 15 seconds.

An example of pseudo-code for an expression may be:

on_event (type, data)    old_data = fetch (type)    new_data =old_data + data    put (type, new_data)

The pseudo-code above may be applied to each displayed variable to counteach occurrence of the variable for each time slice in the time series.In such a transformation, the displayed data may grow over time.

Because the transformations may include user-supplied code, thetransformations may undergo a malware check prior to deployment. Themalware check may attempt to catch malicious or malformedtransformations so that the transformations may not cause unwantederrors or malicious effects.

FIG. 14 is a diagram of an embodiment 1400 showing a computingenvironment that may collect and display trace data in a graph.Embodiment 1400 illustrates hardware components that may deliver theoperations described in embodiment 1300, as well as other embodiments.

The diagram of FIG. 14 illustrates functional components of a system. Insome cases, the component may be a hardware component, a softwarecomponent, or a combination of hardware and software. Some of thecomponents may be application level software, while other components maybe execution environment level components. In some cases, the connectionof one component to another may be a close connection where two or morecomponents are operating on a single hardware platform. In other cases,the connections may be made over network connections spanning longdistances. Each embodiment may use different hardware, software, andinterconnection architectures to achieve the functions described.

Each of the various devices illustrated in embodiment 1400 may have ahardware platform. The respective hardware platforms may be similar tothe hardware platform 204. The devices may be any type of hardwareplatform, such as a personal computer, server computer, game console,tablet computer, mobile telephone, or any other device with aprogrammable processor.

Embodiment 1400 illustrates a network environment in whichtransformations may be deployed to modify the operations of datacollection, storage, and visualizations. The transformations may bestored and deployed in various contexts and managed through atransformation manager.

The environment may include a visualization system 1402, a controllerdevice 1406, a transformation manager 1408, a tracer device 1410, and adata repository 1412. The visualization system 1402 may provide a userinterface for the overall system, and may send commands to thecontroller device 1406 to provide data for a visualization. Thetransformation manager 1408 may receive, store, test, and dispatchtransformations to various devices. The tracer device 1410 may collecttrace data, which may be stored by the data repository 1412.

The visualization system 1402 may contain a hardware platform 1414 onwhich a browser 1416 may run. The browser may present a user interface1418 to a user. The browser 1416 may execute a visualizer 1420, whichmay create and display a graph. The visualizer 1420 may be executablecode that runs within the browser 1416 to retrieve data and render agraph. The visualizer 1420 may include animation routines as well asinteractive components that may allow a user to interact with the graph.

The browser 1416 may also include an editor 1422 through which a usermay enter executable code that may be used as various transformationswithin the larger system. The transformations may be used by a tracerduring data gathering, by a storage manager during data storage, and bya preprocessor when preparing data for visualization. The user suppliedcode may enable a wide range of customizable options for a user tocontrol how data may be gathered, stored, and displayed. Such controlmay be useful in scenarios where a user may experiment with differentways of collecting and viewing data.

A controller device 1406 may operate on a hardware platform 1424. A databrowser 1426 may be controlled from the user interface 1418 on thevisualization system 1402. The data browser 1426 may select data sets tobe displayed by the visualizer 1420. Prior to transmitting the data setswith a communications agent 1432, a preprocessor 1428 may apply varioustransformations 1430 to the data.

A tracer device 1410 may operate on a hardware platform 1434 and have aninstrumented execution environment 1436 that may include a tracer 1438.The tracer 1438 may have a configuration 1440 that may define behaviorsfor the tracer 1438, such as what data to collect and under whichconditions the data may be collected.

The tracer device 1410 may also have a set of transformations 1444,which may process the collected data. The transformations 1444 may beapplied prior to storing the data and may be used to aggregate, compact,condense, or otherwise prepare the data for transmission to a datarepository 1412. The transformations 1444 may also perform dataanalysis, including various statistical analysis, comparisons, or anyother operation.

A data repository 1412 may have a hardware platform 1456 on which astorage manager 1458 may operate. The storage manager 1458 may receivedata from various tracer devices and apply transformations 1460 prior tostoring the data 1462. The transformations 1460 may perform manydifferent types of operations prior to storage, including aggregationand compaction, as well as summarizing, comparisons, or otheroperations.

Embodiment 1400 illustrates two locations for applying pre-storagetransformations. One location may be at the tracer device 1410 astransformations 1444 and the other location may be at the datarepository 1412 as transformations 1460. Either location fortransformations may apply changes to the trace data prior to storage.Transformations applied at the tracer device 1410 may applytransformations prior to data transmittal, as such, some of thetransformations 1444 may compact the data or otherwise prepare the datafor transmittal over the network 1464 to the data repository 1412.

A transformation manager 1408 may operate on a hardware platform 1446and may include a transformation manager 1448. The transformationmanager 1448 may receive transformations from a user through thevisualization system 1402, cause the transformations to be dispatched todifferent devices using a dispatcher 1450. The dispatcher 1450 maycommunicate with the various devices that execute transformations,transmit the transformations, and cause the transformations to executeunder specified conditions.

For example, a dispatcher 1450 may deploy a transformation to the tracerdevice 1410 to compact data prior to transmission and a secondtransformation to the data repository 1412 to create summary statisticsprior to storing the data. The dispatcher 1450 may make thetransformations conditional for tracing a specific application 1442during a specific time period, then cause the transformations to beturned off.

The dispatcher 1450 may also cause certain transformations to bedeployed on the controller device 1406 to prepare, filter, or otherwisemodify data that may be displayed in a visualization. In some cases, thetransformations 1430 deployed to the preprocessor 1428 may be deployedin near-real time under user control so that data displayed in avisualization may be quickly changed.

The transformation manager 1448 may receive new or editedtransformations from a user and then use a malware checker 1452 todetermine if the transformation may be incorrect, incomplete, or has thepotential to cause harm. The malware checker 1452 may use various toolsto approve or deny a given transformation. Such tools may include avirus checker, white list, black list, or other technologies.

The transformation manager 1448 may store transformations in arepository 1454. The stored transformations in the repository 1454 maybe made available as selectable options within the browser 1416.

FIG. 15 is a flowchart illustration of an embodiment 1500 showing amethod for controlling a visualization for a time series of data sets.Embodiment 1500 illustrates the operations of a visualizer and userinterface 1502 in the left hand column and a data browser 1504 in theright hand column.

Other embodiments may use different sequencing, additional or fewersteps, and different nomenclature or terminology to accomplish similarfunctions. In some embodiments, various operations or set of operationsmay be performed in parallel with other operations, either in asynchronous or asynchronous manner. The steps selected here were chosento illustrate some principles of operations in a simplified form.

Embodiment 1500 may illustrate a simplified example of the interactionsbetween a user interface 1502 and a data browser 1504, where the databrowser may process data sets and present the data sets forvisualization. The visualizer may have a data binding or otherconnection to the data browser such that the visualizer may retrieve anddisplay whatever data sets are being presented.

The data browser 1504 may present data sets in sequence so that thevisualizer presents a graph that changes over time. Controls on the userinterface may direct the data browser 1504 to present differentsequences of data sets for normal playback, reverse playback, fastforward, and other sequences.

Embodiment 1500 illustrates a method where a sequence may be defined forpresentation, then the data browser may advance through the sequence tocause data sets to be displayed. In embodiment 1500, the sequences maybe normal forward play where the data sets may be displayed in a timesequence, as well as reverse where the sequence of data sets areinverted or reversed, and fast forward where the sequence only showsevery other data set such that the graph may be updated twice as fast asnormal playback.

Once the sequence is defined, the data browser may use the sequence tolook up the next data set, prepare the data set for viewing, and makethe data set available to the visualizer. Using a data binding or otherconnection, the visualizer may gather the data set and update the graph.

In several of the embodiments presented above, a visualizer may operateon one device and a data browser may operate on a second device. In somecases, both the visualizer and user interface 1502 and data browser 1504may operate on the same device or different devices.

From the user interface 1502, a command may be sent to startvisualization in block 1506. The command may be received by the databrowser 1504 in block 1508.

The sequence to display may be defined in block 1510. For a normalplayback, the sequence may be a time series of data sets in a normal,forward sequence. The next time point to display may be selected inblock 1512, and the data set associated with the time point may beretrieved in block 1514. In some cases, the data set may be retrievedfrom a data repository, which may be a remote device accessed over anetwork.

After retrieving the data set in block 1514, any transformations may beapplied in block 1516 and the data set may be transmitted in block 1518.The process may return to block 1512 to select the next data set in thesequence.

The visualizer and user interface 1502 may receive the new data set inblock 1522 and render or update the graph in block 1524. The visualizermay cycle through the loop of blocks 1522 and 1524 each time the dataset may be updated by the data browser 1504.

Similarly, the data browser 1504 may loop through the blocks 1512through 1518 to fetch the next data set in sequence, prepare the dataset, and make the data set available for the visualizer. The timing ofthe loop of blocks 1512 through 1518 may be set to correspond with thereal time represented by the data sets and thereby cause the graph toupdate in the same time frame as the underlying data.

In some embodiments, the loop of blocks 1512 through 1518 may beadjusted faster or slower so that the playback may be increased ordecreased in speed. In some cases, the data collection frequency may bemuch faster than the playback frequency, which may cause the playback tobe slower than real time. In other cases, the data collection frequencymay be much slower than the playback frequency, causing the playback tobe much faster than real time.

At some point, the user interface 1502 may issue a rewind command inblock 1526, which may be transmitted to the data browser 1504 in block1528. The data browser 1504 may define a new sequence with the timepoints in reverse order in block 1530. The data browser 1504 may returnto block 1512 to select the next data set in the sequence. Because thesequence is now reversed, the data browser 1504 may present the datasets in reverse sequence, and each time the data set may be updated, thevisualizer may update the graph.

A pause command may be issued from the user interface 1502 in block 1532and transmitted to the data browser 1504, which may receive the pausecommand in block 1534. The data browser 1504 may merely stop sendingdata sets in block 1536 to cause the graph from being updated.

A play command may be issued from the user interface 1502 in block 1538and transmitted to the data browser 1504, which may receive the playcommand in block 1540. The data browser 1504 may define a new sequencewith the time points arranged in a forward order in block 1542 andresume sending data sets in block 1544, then continue with block 1512.

A fast forward command may be issued from the user interface 1502 inblock 1546 and transmitted to the data browser 1504, which may receivethe fast forward command in block 1548. The data browser 1504 may createa sequence in block 1550 that has only a subset of the available datasets. In a case where the fast forward may be replayed at twice thenormal play speed, the sequence may include only every other data set.The process may return to block 1512 to cycle through the sequence ofdata sets.

The foregoing description of the subject matter has been presented forpurposes of illustration and description. It is not intended to beexhaustive or to limit the subject matter to the precise form disclosed,and other modifications and variations may be possible in light of theabove teachings. The embodiment was chosen and described in order tobest explain the principles of the invention and its practicalapplication to thereby enable others skilled in the art to best utilizethe invention in various embodiments and various modifications as aresuited to the particular use contemplated. It is intended that theappended claims be construed to include other alternative embodimentsexcept insofar as limited by the prior art.

What is claimed is:
 1. A method comprising: receiving graph data, saidgraph data comprising periodic updates transmitted at a predefinedinterval; displaying said graph data on a display device in a graph;receiving a transformation definition, said transformation definitioncomprising executable code that performs operations on said graph data;performing a malware analysis of said transformation definition;executing said executable code using said graph data to generatetransformed graph data; and displaying said transformed graph data. 2.The method of claim 1 further comprising: presenting a user interface onsaid display device to accept said transformation definition.
 3. Themethod of claim 2, said user interface comprising a set of executionsettings.
 4. The method of claim 3 further comprising storing saidtransformation definition.
 5. The method of claim 4, said user interfacefurther comprising a retrieval mechanism to select said transformationdefinition from a stored set of transformation definitions.
 6. Themethod of claim 1, said graph data being trace data being generatedwhile tracing an application.
 7. The method of claim 6, saidtransformation definition comprising selecting a first tracer output andapplying a transformation function to said first tracer output.
 8. Themethod of claim 7, said first tracer output being comprised in saidtrace data and not comprised in said graph data.
 9. The method of claim7, said transformation function being applied to a plurality of traceroutput data items.
 10. The method of claim 1, said live graph being ananimated graph that changes as each update is received.
 11. The methodof claim 1, said transformed graph data being displayed in place of saidgraph data in said graph.
 12. The method of claim 1, said malwareanalysis comprising checking said executable code against a whitelist.13. The method of claim 1, said malware analysis comprising checkingsaid executable code against a blacklist.
 14. A system comprising: adisplay device; a visualizer that: receives graph data, said graph datacomprising periodic updates and representing a live data stream;displays said graph data on said display device in a graph, said graphbeing dynamically updated with each of said periodic updates; displays auser interface comprising an input for a transformation definition, saidtransformation definition comprising executable code that performsoperations on said graph data; transmits said transformation definitionto a data processing pipeline, said data processing pipeline having atransformation evaluator and producing said graph data; receivingtransformed graph data, said transformed graph data being processedaccording to said transformation definition; and updating said graphusing said transformed graph data.
 15. The system of claim 14, said userinterface comprising an editor that edits said executable code.
 16. Thesystem of claim 15, said user interface comprising a retrieval mechanismto select said transformation definition from a stored state.
 17. Thesystem of claim 16, said user interface allowing editing of saidtransformation definition after retrieval from said stored state. 18.The system of claim 14 further comprising: a storage repositorycomprising a plurality of stored transformation definitions.
 19. Thesystem of claim 18 further comprising: a user interface componentcomprising a mechanism to recall said stored transformations.
 20. Thesystem of claim 14, said data processing pipeline being contained withinsaid system.